{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tutorial 3: Training a WSI Classification Model with ABMIL\n",
    "\n",
    "This tutorial will guide you step-by-step to train an attention-based multiple instance learning model using Trident patch embeddings. Then, we will generation attention heatmaps using the pretrained model. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### A- Installation and patch feature extraction using UNI\n",
    "\n",
    "#### Step 1: Download a dataset of whole-slide images\n",
    "\n",
    "You can use your own WSIs or download a publicly available dataset, e.g. from:\n",
    "\n",
    "- **CPTAC CCRCC WSIs**: Download from the [TCIA Cancer Imaging Archive](https://www.cancerimagingarchive.net/collection/cptac-ccrcc/).\n",
    "- **Store WSIs**: Save all WSIs into a local directory, e.g.,  \n",
    "  ```bash\n",
    "  ./CPTAC-CCRCC_v1/CCRCC\n",
    "  ```\n",
    "\n",
    "#### Step 2:  Run CONCH v1.5 feature extraction:\n",
    "\n",
    "Navigate to the base directory of Trident and execute the following command:\n",
    "\n",
    "```bash\n",
    "python run_batch_of_slides.py --task all \\\n",
    "  --wsi_dir ./CPTAC-CCRCC_v1/CCRCC \\\n",
    "  --job_dir ./tutorial-3 \\\n",
    "  --patch_encoder conch_v15 \\\n",
    "  --mag 20 \\\n",
    "  --patch_size 512\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### B- Download labels with data splits\n",
    "\n",
    "Here, we use Patho-Bench CPTAC-CCRCC labels for predicting BAP1 mutation. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/requests/__init__.py:86: RequestsDependencyWarning: Unable to find acceptable character detection dependency (chardet or charset_normalizer).\n",
      "  warnings.warn(\n",
      "'(MaxRetryError(\"HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /datasets/MahmoodLab/Patho-Bench/resolve/main/README.md (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x767d041b8b20>: Failed to establish a new connection: [Errno 101] Network is unreachable'))\"), '(Request ID: 09274cf1-6be3-4ad4-9fe4-43411b7f8701)')' thrown while requesting HEAD https://huggingface.co/datasets/MahmoodLab/Patho-Bench/resolve/main/README.md\n",
      "Retrying in 1s [Retry 1/5].\n",
      "'(MaxRetryError(\"HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /datasets/MahmoodLab/Patho-Bench/resolve/main/README.md (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x767d041ba650>: Failed to establish a new connection: [Errno 101] Network is unreachable'))\"), '(Request ID: 67a74941-19f0-46ee-9e27-aeb186cbfc67)')' thrown while requesting HEAD https://huggingface.co/datasets/MahmoodLab/Patho-Bench/resolve/main/README.md\n",
      "Retrying in 2s [Retry 2/5].\n"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "\u001b[1;32m/data0/lcy/trident/tutorials/3-1-ABMIL-Classification.ipynb 单元格 4\u001b[0m line \u001b[0;36m5\n\u001b[1;32m      <a href='vscode-notebook-cell://ssh-remote%2Bnfy4090/data0/lcy/trident/tutorials/3-1-ABMIL-Classification.ipynb#W3sdnNjb2RlLXJlbW90ZQ%3D%3D?line=1'>2</a>\u001b[0m \u001b[39mimport\u001b[39;00m\u001b[39m \u001b[39m\u001b[39mpandas\u001b[39;00m\u001b[39m \u001b[39m\u001b[39mas\u001b[39;00m\u001b[39m \u001b[39m\u001b[39mpd\u001b[39;00m\n\u001b[1;32m      <a href='vscode-notebook-cell://ssh-remote%2Bnfy4090/data0/lcy/trident/tutorials/3-1-ABMIL-Classification.ipynb#W3sdnNjb2RlLXJlbW90ZQ%3D%3D?line=3'>4</a>\u001b[0m \u001b[39m# Download labels as csv\u001b[39;00m\n\u001b[0;32m----> <a href='vscode-notebook-cell://ssh-remote%2Bnfy4090/data0/lcy/trident/tutorials/3-1-ABMIL-Classification.ipynb#W3sdnNjb2RlLXJlbW90ZQ%3D%3D?line=4'>5</a>\u001b[0m datasets\u001b[39m.\u001b[39;49mload_dataset(\n\u001b[1;32m      <a href='vscode-notebook-cell://ssh-remote%2Bnfy4090/data0/lcy/trident/tutorials/3-1-ABMIL-Classification.ipynb#W3sdnNjb2RlLXJlbW90ZQ%3D%3D?line=5'>6</a>\u001b[0m     \u001b[39m'\u001b[39;49m\u001b[39mMahmoodLab/Patho-Bench\u001b[39;49m\u001b[39m'\u001b[39;49m, \n\u001b[1;32m      <a href='vscode-notebook-cell://ssh-remote%2Bnfy4090/data0/lcy/trident/tutorials/3-1-ABMIL-Classification.ipynb#W3sdnNjb2RlLXJlbW90ZQ%3D%3D?line=6'>7</a>\u001b[0m     cache_dir\u001b[39m=\u001b[39;49m\u001b[39m'\u001b[39;49m\u001b[39m./tutorial-3\u001b[39;49m\u001b[39m'\u001b[39;49m,\n\u001b[1;32m      <a href='vscode-notebook-cell://ssh-remote%2Bnfy4090/data0/lcy/trident/tutorials/3-1-ABMIL-Classification.ipynb#W3sdnNjb2RlLXJlbW90ZQ%3D%3D?line=7'>8</a>\u001b[0m     dataset_to_download\u001b[39m=\u001b[39;49m\u001b[39m'\u001b[39;49m\u001b[39mcptac_ccrcc\u001b[39;49m\u001b[39m'\u001b[39;49m,     \n\u001b[1;32m      <a href='vscode-notebook-cell://ssh-remote%2Bnfy4090/data0/lcy/trident/tutorials/3-1-ABMIL-Classification.ipynb#W3sdnNjb2RlLXJlbW90ZQ%3D%3D?line=8'>9</a>\u001b[0m     task_in_dataset\u001b[39m=\u001b[39;49m\u001b[39m'\u001b[39;49m\u001b[39mBAP1_mutation\u001b[39;49m\u001b[39m'\u001b[39;49m,           \n\u001b[1;32m     <a href='vscode-notebook-cell://ssh-remote%2Bnfy4090/data0/lcy/trident/tutorials/3-1-ABMIL-Classification.ipynb#W3sdnNjb2RlLXJlbW90ZQ%3D%3D?line=9'>10</a>\u001b[0m     trust_remote_code\u001b[39m=\u001b[39;49m\u001b[39mTrue\u001b[39;49;00m\n\u001b[1;32m     <a href='vscode-notebook-cell://ssh-remote%2Bnfy4090/data0/lcy/trident/tutorials/3-1-ABMIL-Classification.ipynb#W3sdnNjb2RlLXJlbW90ZQ%3D%3D?line=10'>11</a>\u001b[0m )\n\u001b[1;32m     <a href='vscode-notebook-cell://ssh-remote%2Bnfy4090/data0/lcy/trident/tutorials/3-1-ABMIL-Classification.ipynb#W3sdnNjb2RlLXJlbW90ZQ%3D%3D?line=12'>13</a>\u001b[0m \u001b[39m# Visualize my labels and splits\u001b[39;00m\n\u001b[1;32m     <a href='vscode-notebook-cell://ssh-remote%2Bnfy4090/data0/lcy/trident/tutorials/3-1-ABMIL-Classification.ipynb#W3sdnNjb2RlLXJlbW90ZQ%3D%3D?line=13'>14</a>\u001b[0m df \u001b[39m=\u001b[39m pd\u001b[39m.\u001b[39mread_csv(\u001b[39m'\u001b[39m\u001b[39mtutorial-3/cptac_ccrcc/BAP1_mutation/k=all.tsv\u001b[39m\u001b[39m'\u001b[39m, sep\u001b[39m=\u001b[39m\u001b[39m\"\u001b[39m\u001b[39m\\t\u001b[39;00m\u001b[39m\"\u001b[39m)\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/datasets/load.py:2062\u001b[0m, in \u001b[0;36mload_dataset\u001b[0;34m(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, verification_mode, keep_in_memory, save_infos, revision, token, streaming, num_proc, storage_options, trust_remote_code, **config_kwargs)\u001b[0m\n\u001b[1;32m   2057\u001b[0m verification_mode \u001b[39m=\u001b[39m VerificationMode(\n\u001b[1;32m   2058\u001b[0m     (verification_mode \u001b[39mor\u001b[39;00m VerificationMode\u001b[39m.\u001b[39mBASIC_CHECKS) \u001b[39mif\u001b[39;00m \u001b[39mnot\u001b[39;00m save_infos \u001b[39melse\u001b[39;00m VerificationMode\u001b[39m.\u001b[39mALL_CHECKS\n\u001b[1;32m   2059\u001b[0m )\n\u001b[1;32m   2061\u001b[0m \u001b[39m# Create a dataset builder\u001b[39;00m\n\u001b[0;32m-> 2062\u001b[0m builder_instance \u001b[39m=\u001b[39m load_dataset_builder(\n\u001b[1;32m   2063\u001b[0m     path\u001b[39m=\u001b[39;49mpath,\n\u001b[1;32m   2064\u001b[0m     name\u001b[39m=\u001b[39;49mname,\n\u001b[1;32m   2065\u001b[0m     data_dir\u001b[39m=\u001b[39;49mdata_dir,\n\u001b[1;32m   2066\u001b[0m     data_files\u001b[39m=\u001b[39;49mdata_files,\n\u001b[1;32m   2067\u001b[0m     cache_dir\u001b[39m=\u001b[39;49mcache_dir,\n\u001b[1;32m   2068\u001b[0m     features\u001b[39m=\u001b[39;49mfeatures,\n\u001b[1;32m   2069\u001b[0m     download_config\u001b[39m=\u001b[39;49mdownload_config,\n\u001b[1;32m   2070\u001b[0m     download_mode\u001b[39m=\u001b[39;49mdownload_mode,\n\u001b[1;32m   2071\u001b[0m     revision\u001b[39m=\u001b[39;49mrevision,\n\u001b[1;32m   2072\u001b[0m     token\u001b[39m=\u001b[39;49mtoken,\n\u001b[1;32m   2073\u001b[0m     storage_options\u001b[39m=\u001b[39;49mstorage_options,\n\u001b[1;32m   2074\u001b[0m     trust_remote_code\u001b[39m=\u001b[39;49mtrust_remote_code,\n\u001b[1;32m   2075\u001b[0m     _require_default_config_name\u001b[39m=\u001b[39;49mname \u001b[39mis\u001b[39;49;00m \u001b[39mNone\u001b[39;49;00m,\n\u001b[1;32m   2076\u001b[0m     \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mconfig_kwargs,\n\u001b[1;32m   2077\u001b[0m )\n\u001b[1;32m   2079\u001b[0m \u001b[39m# Return iterable dataset in case of streaming\u001b[39;00m\n\u001b[1;32m   2080\u001b[0m \u001b[39mif\u001b[39;00m streaming:\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/datasets/load.py:1782\u001b[0m, in \u001b[0;36mload_dataset_builder\u001b[0;34m(path, name, data_dir, data_files, cache_dir, features, download_config, download_mode, revision, token, storage_options, trust_remote_code, _require_default_config_name, **config_kwargs)\u001b[0m\n\u001b[1;32m   1780\u001b[0m     download_config \u001b[39m=\u001b[39m download_config\u001b[39m.\u001b[39mcopy() \u001b[39mif\u001b[39;00m download_config \u001b[39melse\u001b[39;00m DownloadConfig()\n\u001b[1;32m   1781\u001b[0m     download_config\u001b[39m.\u001b[39mstorage_options\u001b[39m.\u001b[39mupdate(storage_options)\n\u001b[0;32m-> 1782\u001b[0m dataset_module \u001b[39m=\u001b[39m dataset_module_factory(\n\u001b[1;32m   1783\u001b[0m     path,\n\u001b[1;32m   1784\u001b[0m     revision\u001b[39m=\u001b[39;49mrevision,\n\u001b[1;32m   1785\u001b[0m     download_config\u001b[39m=\u001b[39;49mdownload_config,\n\u001b[1;32m   1786\u001b[0m     download_mode\u001b[39m=\u001b[39;49mdownload_mode,\n\u001b[1;32m   1787\u001b[0m     data_dir\u001b[39m=\u001b[39;49mdata_dir,\n\u001b[1;32m   1788\u001b[0m     data_files\u001b[39m=\u001b[39;49mdata_files,\n\u001b[1;32m   1789\u001b[0m     cache_dir\u001b[39m=\u001b[39;49mcache_dir,\n\u001b[1;32m   1790\u001b[0m     trust_remote_code\u001b[39m=\u001b[39;49mtrust_remote_code,\n\u001b[1;32m   1791\u001b[0m     _require_default_config_name\u001b[39m=\u001b[39;49m_require_default_config_name,\n\u001b[1;32m   1792\u001b[0m     _require_custom_configs\u001b[39m=\u001b[39;49m\u001b[39mbool\u001b[39;49m(config_kwargs),\n\u001b[1;32m   1793\u001b[0m )\n\u001b[1;32m   1794\u001b[0m \u001b[39m# Get dataset builder class from the processing script\u001b[39;00m\n\u001b[1;32m   1795\u001b[0m builder_kwargs \u001b[39m=\u001b[39m dataset_module\u001b[39m.\u001b[39mbuilder_kwargs\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/datasets/load.py:1534\u001b[0m, in \u001b[0;36mdataset_module_factory\u001b[0;34m(path, revision, download_config, download_mode, dynamic_modules_path, data_dir, data_files, cache_dir, trust_remote_code, _require_default_config_name, _require_custom_configs, **download_kwargs)\u001b[0m\n\u001b[1;32m   1532\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[1;32m   1533\u001b[0m     _raise_if_offline_mode_is_enabled()\n\u001b[0;32m-> 1534\u001b[0m     dataset_readme_path \u001b[39m=\u001b[39m api\u001b[39m.\u001b[39;49mhf_hub_download(\n\u001b[1;32m   1535\u001b[0m         repo_id\u001b[39m=\u001b[39;49mpath,\n\u001b[1;32m   1536\u001b[0m         filename\u001b[39m=\u001b[39;49mconfig\u001b[39m.\u001b[39;49mREPOCARD_FILENAME,\n\u001b[1;32m   1537\u001b[0m         repo_type\u001b[39m=\u001b[39;49m\u001b[39m\"\u001b[39;49m\u001b[39mdataset\u001b[39;49m\u001b[39m\"\u001b[39;49m,\n\u001b[1;32m   1538\u001b[0m         revision\u001b[39m=\u001b[39;49mrevision,\n\u001b[1;32m   1539\u001b[0m         proxies\u001b[39m=\u001b[39;49mdownload_config\u001b[39m.\u001b[39;49mproxies,\n\u001b[1;32m   1540\u001b[0m     )\n\u001b[1;32m   1541\u001b[0m     commit_hash \u001b[39m=\u001b[39m os\u001b[39m.\u001b[39mpath\u001b[39m.\u001b[39mbasename(os\u001b[39m.\u001b[39mpath\u001b[39m.\u001b[39mdirname(dataset_readme_path))\n\u001b[1;32m   1542\u001b[0m \u001b[39mexcept\u001b[39;00m LocalEntryNotFoundError \u001b[39mas\u001b[39;00m e:\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py:114\u001b[0m, in \u001b[0;36mvalidate_hf_hub_args.<locals>._inner_fn\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    111\u001b[0m \u001b[39mif\u001b[39;00m check_use_auth_token:\n\u001b[1;32m    112\u001b[0m     kwargs \u001b[39m=\u001b[39m smoothly_deprecate_use_auth_token(fn_name\u001b[39m=\u001b[39mfn\u001b[39m.\u001b[39m\u001b[39m__name__\u001b[39m, has_token\u001b[39m=\u001b[39mhas_token, kwargs\u001b[39m=\u001b[39mkwargs)\n\u001b[0;32m--> 114\u001b[0m \u001b[39mreturn\u001b[39;00m fn(\u001b[39m*\u001b[39;49margs, \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/huggingface_hub/hf_api.py:5539\u001b[0m, in \u001b[0;36mHfApi.hf_hub_download\u001b[0;34m(self, repo_id, filename, subfolder, repo_type, revision, cache_dir, local_dir, force_download, proxies, etag_timeout, token, local_files_only, resume_download, force_filename, local_dir_use_symlinks)\u001b[0m\n\u001b[1;32m   5535\u001b[0m \u001b[39mif\u001b[39;00m token \u001b[39mis\u001b[39;00m \u001b[39mNone\u001b[39;00m:\n\u001b[1;32m   5536\u001b[0m     \u001b[39m# Cannot do `token = token or self.token` as token can be `False`.\u001b[39;00m\n\u001b[1;32m   5537\u001b[0m     token \u001b[39m=\u001b[39m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mtoken\n\u001b[0;32m-> 5539\u001b[0m \u001b[39mreturn\u001b[39;00m hf_hub_download(\n\u001b[1;32m   5540\u001b[0m     repo_id\u001b[39m=\u001b[39;49mrepo_id,\n\u001b[1;32m   5541\u001b[0m     filename\u001b[39m=\u001b[39;49mfilename,\n\u001b[1;32m   5542\u001b[0m     subfolder\u001b[39m=\u001b[39;49msubfolder,\n\u001b[1;32m   5543\u001b[0m     repo_type\u001b[39m=\u001b[39;49mrepo_type,\n\u001b[1;32m   5544\u001b[0m     revision\u001b[39m=\u001b[39;49mrevision,\n\u001b[1;32m   5545\u001b[0m     endpoint\u001b[39m=\u001b[39;49m\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mendpoint,\n\u001b[1;32m   5546\u001b[0m     library_name\u001b[39m=\u001b[39;49m\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mlibrary_name,\n\u001b[1;32m   5547\u001b[0m     library_version\u001b[39m=\u001b[39;49m\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mlibrary_version,\n\u001b[1;32m   5548\u001b[0m     cache_dir\u001b[39m=\u001b[39;49mcache_dir,\n\u001b[1;32m   5549\u001b[0m     local_dir\u001b[39m=\u001b[39;49mlocal_dir,\n\u001b[1;32m   5550\u001b[0m     local_dir_use_symlinks\u001b[39m=\u001b[39;49mlocal_dir_use_symlinks,\n\u001b[1;32m   5551\u001b[0m     user_agent\u001b[39m=\u001b[39;49m\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49muser_agent,\n\u001b[1;32m   5552\u001b[0m     force_download\u001b[39m=\u001b[39;49mforce_download,\n\u001b[1;32m   5553\u001b[0m     force_filename\u001b[39m=\u001b[39;49mforce_filename,\n\u001b[1;32m   5554\u001b[0m     proxies\u001b[39m=\u001b[39;49mproxies,\n\u001b[1;32m   5555\u001b[0m     etag_timeout\u001b[39m=\u001b[39;49metag_timeout,\n\u001b[1;32m   5556\u001b[0m     resume_download\u001b[39m=\u001b[39;49mresume_download,\n\u001b[1;32m   5557\u001b[0m     token\u001b[39m=\u001b[39;49mtoken,\n\u001b[1;32m   5558\u001b[0m     headers\u001b[39m=\u001b[39;49m\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mheaders,\n\u001b[1;32m   5559\u001b[0m     local_files_only\u001b[39m=\u001b[39;49mlocal_files_only,\n\u001b[1;32m   5560\u001b[0m )\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py:114\u001b[0m, in \u001b[0;36mvalidate_hf_hub_args.<locals>._inner_fn\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    111\u001b[0m \u001b[39mif\u001b[39;00m check_use_auth_token:\n\u001b[1;32m    112\u001b[0m     kwargs \u001b[39m=\u001b[39m smoothly_deprecate_use_auth_token(fn_name\u001b[39m=\u001b[39mfn\u001b[39m.\u001b[39m\u001b[39m__name__\u001b[39m, has_token\u001b[39m=\u001b[39mhas_token, kwargs\u001b[39m=\u001b[39mkwargs)\n\u001b[0;32m--> 114\u001b[0m \u001b[39mreturn\u001b[39;00m fn(\u001b[39m*\u001b[39;49margs, \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/huggingface_hub/file_download.py:1010\u001b[0m, in \u001b[0;36mhf_hub_download\u001b[0;34m(repo_id, filename, subfolder, repo_type, revision, library_name, library_version, cache_dir, local_dir, user_agent, force_download, proxies, etag_timeout, token, local_files_only, headers, endpoint, resume_download, force_filename, local_dir_use_symlinks)\u001b[0m\n\u001b[1;32m    990\u001b[0m     \u001b[39mreturn\u001b[39;00m _hf_hub_download_to_local_dir(\n\u001b[1;32m    991\u001b[0m         \u001b[39m# Destination\u001b[39;00m\n\u001b[1;32m    992\u001b[0m         local_dir\u001b[39m=\u001b[39mlocal_dir,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m   1007\u001b[0m         local_files_only\u001b[39m=\u001b[39mlocal_files_only,\n\u001b[1;32m   1008\u001b[0m     )\n\u001b[1;32m   1009\u001b[0m \u001b[39melse\u001b[39;00m:\n\u001b[0;32m-> 1010\u001b[0m     \u001b[39mreturn\u001b[39;00m _hf_hub_download_to_cache_dir(\n\u001b[1;32m   1011\u001b[0m         \u001b[39m# Destination\u001b[39;49;00m\n\u001b[1;32m   1012\u001b[0m         cache_dir\u001b[39m=\u001b[39;49mcache_dir,\n\u001b[1;32m   1013\u001b[0m         \u001b[39m# File info\u001b[39;49;00m\n\u001b[1;32m   1014\u001b[0m         repo_id\u001b[39m=\u001b[39;49mrepo_id,\n\u001b[1;32m   1015\u001b[0m         filename\u001b[39m=\u001b[39;49mfilename,\n\u001b[1;32m   1016\u001b[0m         repo_type\u001b[39m=\u001b[39;49mrepo_type,\n\u001b[1;32m   1017\u001b[0m         revision\u001b[39m=\u001b[39;49mrevision,\n\u001b[1;32m   1018\u001b[0m         \u001b[39m# HTTP info\u001b[39;49;00m\n\u001b[1;32m   1019\u001b[0m         endpoint\u001b[39m=\u001b[39;49mendpoint,\n\u001b[1;32m   1020\u001b[0m         etag_timeout\u001b[39m=\u001b[39;49metag_timeout,\n\u001b[1;32m   1021\u001b[0m         headers\u001b[39m=\u001b[39;49mhf_headers,\n\u001b[1;32m   1022\u001b[0m         proxies\u001b[39m=\u001b[39;49mproxies,\n\u001b[1;32m   1023\u001b[0m         token\u001b[39m=\u001b[39;49mtoken,\n\u001b[1;32m   1024\u001b[0m         \u001b[39m# Additional options\u001b[39;49;00m\n\u001b[1;32m   1025\u001b[0m         local_files_only\u001b[39m=\u001b[39;49mlocal_files_only,\n\u001b[1;32m   1026\u001b[0m         force_download\u001b[39m=\u001b[39;49mforce_download,\n\u001b[1;32m   1027\u001b[0m     )\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/huggingface_hub/file_download.py:1073\u001b[0m, in \u001b[0;36m_hf_hub_download_to_cache_dir\u001b[0;34m(cache_dir, repo_id, filename, repo_type, revision, endpoint, etag_timeout, headers, proxies, token, local_files_only, force_download)\u001b[0m\n\u001b[1;32m   1069\u001b[0m         \u001b[39mreturn\u001b[39;00m pointer_path\n\u001b[1;32m   1071\u001b[0m \u001b[39m# Try to get metadata (etag, commit_hash, url, size) from the server.\u001b[39;00m\n\u001b[1;32m   1072\u001b[0m \u001b[39m# If we can't, a HEAD request error is returned.\u001b[39;00m\n\u001b[0;32m-> 1073\u001b[0m (url_to_download, etag, commit_hash, expected_size, xet_file_data, head_call_error) \u001b[39m=\u001b[39m _get_metadata_or_catch_error(\n\u001b[1;32m   1074\u001b[0m     repo_id\u001b[39m=\u001b[39;49mrepo_id,\n\u001b[1;32m   1075\u001b[0m     filename\u001b[39m=\u001b[39;49mfilename,\n\u001b[1;32m   1076\u001b[0m     repo_type\u001b[39m=\u001b[39;49mrepo_type,\n\u001b[1;32m   1077\u001b[0m     revision\u001b[39m=\u001b[39;49mrevision,\n\u001b[1;32m   1078\u001b[0m     endpoint\u001b[39m=\u001b[39;49mendpoint,\n\u001b[1;32m   1079\u001b[0m     proxies\u001b[39m=\u001b[39;49mproxies,\n\u001b[1;32m   1080\u001b[0m     etag_timeout\u001b[39m=\u001b[39;49metag_timeout,\n\u001b[1;32m   1081\u001b[0m     headers\u001b[39m=\u001b[39;49mheaders,\n\u001b[1;32m   1082\u001b[0m     token\u001b[39m=\u001b[39;49mtoken,\n\u001b[1;32m   1083\u001b[0m     local_files_only\u001b[39m=\u001b[39;49mlocal_files_only,\n\u001b[1;32m   1084\u001b[0m     storage_folder\u001b[39m=\u001b[39;49mstorage_folder,\n\u001b[1;32m   1085\u001b[0m     relative_filename\u001b[39m=\u001b[39;49mrelative_filename,\n\u001b[1;32m   1086\u001b[0m )\n\u001b[1;32m   1088\u001b[0m \u001b[39m# etag can be None for several reasons:\u001b[39;00m\n\u001b[1;32m   1089\u001b[0m \u001b[39m# 1. we passed local_files_only.\u001b[39;00m\n\u001b[1;32m   1090\u001b[0m \u001b[39m# 2. we don't have a connection\u001b[39;00m\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m   1096\u001b[0m \u001b[39m# If the specified revision is a commit hash, look inside \"snapshots\".\u001b[39;00m\n\u001b[1;32m   1097\u001b[0m \u001b[39m# If the specified revision is a branch or tag, look inside \"refs\".\u001b[39;00m\n\u001b[1;32m   1098\u001b[0m \u001b[39mif\u001b[39;00m head_call_error \u001b[39mis\u001b[39;00m \u001b[39mnot\u001b[39;00m \u001b[39mNone\u001b[39;00m:\n\u001b[1;32m   1099\u001b[0m     \u001b[39m# Couldn't make a HEAD call => let's try to find a local file\u001b[39;00m\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/huggingface_hub/file_download.py:1546\u001b[0m, in \u001b[0;36m_get_metadata_or_catch_error\u001b[0;34m(repo_id, filename, repo_type, revision, endpoint, proxies, etag_timeout, headers, token, local_files_only, relative_filename, storage_folder)\u001b[0m\n\u001b[1;32m   1544\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[1;32m   1545\u001b[0m     \u001b[39mtry\u001b[39;00m:\n\u001b[0;32m-> 1546\u001b[0m         metadata \u001b[39m=\u001b[39m get_hf_file_metadata(\n\u001b[1;32m   1547\u001b[0m             url\u001b[39m=\u001b[39;49murl, proxies\u001b[39m=\u001b[39;49mproxies, timeout\u001b[39m=\u001b[39;49metag_timeout, headers\u001b[39m=\u001b[39;49mheaders, token\u001b[39m=\u001b[39;49mtoken, endpoint\u001b[39m=\u001b[39;49mendpoint\n\u001b[1;32m   1548\u001b[0m         )\n\u001b[1;32m   1549\u001b[0m     \u001b[39mexcept\u001b[39;00m EntryNotFoundError \u001b[39mas\u001b[39;00m http_error:\n\u001b[1;32m   1550\u001b[0m         \u001b[39mif\u001b[39;00m storage_folder \u001b[39mis\u001b[39;00m \u001b[39mnot\u001b[39;00m \u001b[39mNone\u001b[39;00m \u001b[39mand\u001b[39;00m relative_filename \u001b[39mis\u001b[39;00m \u001b[39mnot\u001b[39;00m \u001b[39mNone\u001b[39;00m:\n\u001b[1;32m   1551\u001b[0m             \u001b[39m# Cache the non-existence of the file\u001b[39;00m\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py:114\u001b[0m, in \u001b[0;36mvalidate_hf_hub_args.<locals>._inner_fn\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    111\u001b[0m \u001b[39mif\u001b[39;00m check_use_auth_token:\n\u001b[1;32m    112\u001b[0m     kwargs \u001b[39m=\u001b[39m smoothly_deprecate_use_auth_token(fn_name\u001b[39m=\u001b[39mfn\u001b[39m.\u001b[39m\u001b[39m__name__\u001b[39m, has_token\u001b[39m=\u001b[39mhas_token, kwargs\u001b[39m=\u001b[39mkwargs)\n\u001b[0;32m--> 114\u001b[0m \u001b[39mreturn\u001b[39;00m fn(\u001b[39m*\u001b[39;49margs, \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/huggingface_hub/file_download.py:1463\u001b[0m, in \u001b[0;36mget_hf_file_metadata\u001b[0;34m(url, token, proxies, timeout, library_name, library_version, user_agent, headers, endpoint)\u001b[0m\n\u001b[1;32m   1460\u001b[0m hf_headers[\u001b[39m\"\u001b[39m\u001b[39mAccept-Encoding\u001b[39m\u001b[39m\"\u001b[39m] \u001b[39m=\u001b[39m \u001b[39m\"\u001b[39m\u001b[39midentity\u001b[39m\u001b[39m\"\u001b[39m  \u001b[39m# prevent any compression => we want to know the real size of the file\u001b[39;00m\n\u001b[1;32m   1462\u001b[0m \u001b[39m# Retrieve metadata\u001b[39;00m\n\u001b[0;32m-> 1463\u001b[0m r \u001b[39m=\u001b[39m _request_wrapper(\n\u001b[1;32m   1464\u001b[0m     method\u001b[39m=\u001b[39;49m\u001b[39m\"\u001b[39;49m\u001b[39mHEAD\u001b[39;49m\u001b[39m\"\u001b[39;49m,\n\u001b[1;32m   1465\u001b[0m     url\u001b[39m=\u001b[39;49murl,\n\u001b[1;32m   1466\u001b[0m     headers\u001b[39m=\u001b[39;49mhf_headers,\n\u001b[1;32m   1467\u001b[0m     allow_redirects\u001b[39m=\u001b[39;49m\u001b[39mFalse\u001b[39;49;00m,\n\u001b[1;32m   1468\u001b[0m     follow_relative_redirects\u001b[39m=\u001b[39;49m\u001b[39mTrue\u001b[39;49;00m,\n\u001b[1;32m   1469\u001b[0m     proxies\u001b[39m=\u001b[39;49mproxies,\n\u001b[1;32m   1470\u001b[0m     timeout\u001b[39m=\u001b[39;49mtimeout,\n\u001b[1;32m   1471\u001b[0m )\n\u001b[1;32m   1472\u001b[0m hf_raise_for_status(r)\n\u001b[1;32m   1474\u001b[0m \u001b[39m# Return\u001b[39;00m\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/huggingface_hub/file_download.py:286\u001b[0m, in \u001b[0;36m_request_wrapper\u001b[0;34m(method, url, follow_relative_redirects, **params)\u001b[0m\n\u001b[1;32m    284\u001b[0m \u001b[39m# Recursively follow relative redirects\u001b[39;00m\n\u001b[1;32m    285\u001b[0m \u001b[39mif\u001b[39;00m follow_relative_redirects:\n\u001b[0;32m--> 286\u001b[0m     response \u001b[39m=\u001b[39m _request_wrapper(\n\u001b[1;32m    287\u001b[0m         method\u001b[39m=\u001b[39;49mmethod,\n\u001b[1;32m    288\u001b[0m         url\u001b[39m=\u001b[39;49murl,\n\u001b[1;32m    289\u001b[0m         follow_relative_redirects\u001b[39m=\u001b[39;49m\u001b[39mFalse\u001b[39;49;00m,\n\u001b[1;32m    290\u001b[0m         \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mparams,\n\u001b[1;32m    291\u001b[0m     )\n\u001b[1;32m    293\u001b[0m     \u001b[39m# If redirection, we redirect only relative paths.\u001b[39;00m\n\u001b[1;32m    294\u001b[0m     \u001b[39m# This is useful in case of a renamed repository.\u001b[39;00m\n\u001b[1;32m    295\u001b[0m     \u001b[39mif\u001b[39;00m \u001b[39m300\u001b[39m \u001b[39m<\u001b[39m\u001b[39m=\u001b[39m response\u001b[39m.\u001b[39mstatus_code \u001b[39m<\u001b[39m\u001b[39m=\u001b[39m \u001b[39m399\u001b[39m:\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/huggingface_hub/file_download.py:309\u001b[0m, in \u001b[0;36m_request_wrapper\u001b[0;34m(method, url, follow_relative_redirects, **params)\u001b[0m\n\u001b[1;32m    306\u001b[0m     \u001b[39mreturn\u001b[39;00m response\n\u001b[1;32m    308\u001b[0m \u001b[39m# Perform request and return if status_code is not in the retry list.\u001b[39;00m\n\u001b[0;32m--> 309\u001b[0m response \u001b[39m=\u001b[39m http_backoff(method\u001b[39m=\u001b[39;49mmethod, url\u001b[39m=\u001b[39;49murl, \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mparams)\n\u001b[1;32m    310\u001b[0m hf_raise_for_status(response)\n\u001b[1;32m    311\u001b[0m \u001b[39mreturn\u001b[39;00m response\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/huggingface_hub/utils/_http.py:308\u001b[0m, in \u001b[0;36mhttp_backoff\u001b[0;34m(method, url, max_retries, base_wait_time, max_wait_time, retry_on_exceptions, retry_on_status_codes, **kwargs)\u001b[0m\n\u001b[1;32m    305\u001b[0m     kwargs[\u001b[39m\"\u001b[39m\u001b[39mdata\u001b[39m\u001b[39m\"\u001b[39m]\u001b[39m.\u001b[39mseek(io_obj_initial_pos)\n\u001b[1;32m    307\u001b[0m \u001b[39m# Perform request and return if status_code is not in the retry list.\u001b[39;00m\n\u001b[0;32m--> 308\u001b[0m response \u001b[39m=\u001b[39m session\u001b[39m.\u001b[39;49mrequest(method\u001b[39m=\u001b[39;49mmethod, url\u001b[39m=\u001b[39;49murl, \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n\u001b[1;32m    309\u001b[0m \u001b[39mif\u001b[39;00m response\u001b[39m.\u001b[39mstatus_code \u001b[39mnot\u001b[39;00m \u001b[39min\u001b[39;00m retry_on_status_codes:\n\u001b[1;32m    310\u001b[0m     \u001b[39mreturn\u001b[39;00m response\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/requests/sessions.py:589\u001b[0m, in \u001b[0;36mSession.request\u001b[0;34m(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)\u001b[0m\n\u001b[1;32m    584\u001b[0m send_kwargs \u001b[39m=\u001b[39m {\n\u001b[1;32m    585\u001b[0m     \u001b[39m\"\u001b[39m\u001b[39mtimeout\u001b[39m\u001b[39m\"\u001b[39m: timeout,\n\u001b[1;32m    586\u001b[0m     \u001b[39m\"\u001b[39m\u001b[39mallow_redirects\u001b[39m\u001b[39m\"\u001b[39m: allow_redirects,\n\u001b[1;32m    587\u001b[0m }\n\u001b[1;32m    588\u001b[0m send_kwargs\u001b[39m.\u001b[39mupdate(settings)\n\u001b[0;32m--> 589\u001b[0m resp \u001b[39m=\u001b[39m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49msend(prep, \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49msend_kwargs)\n\u001b[1;32m    591\u001b[0m \u001b[39mreturn\u001b[39;00m resp\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/requests/sessions.py:703\u001b[0m, in \u001b[0;36mSession.send\u001b[0;34m(self, request, **kwargs)\u001b[0m\n\u001b[1;32m    700\u001b[0m start \u001b[39m=\u001b[39m preferred_clock()\n\u001b[1;32m    702\u001b[0m \u001b[39m# Send the request\u001b[39;00m\n\u001b[0;32m--> 703\u001b[0m r \u001b[39m=\u001b[39m adapter\u001b[39m.\u001b[39;49msend(request, \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n\u001b[1;32m    705\u001b[0m \u001b[39m# Total elapsed time of the request (approximately)\u001b[39;00m\n\u001b[1;32m    706\u001b[0m elapsed \u001b[39m=\u001b[39m preferred_clock() \u001b[39m-\u001b[39m start\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/huggingface_hub/utils/_http.py:95\u001b[0m, in \u001b[0;36mUniqueRequestIdAdapter.send\u001b[0;34m(self, request, *args, **kwargs)\u001b[0m\n\u001b[1;32m     93\u001b[0m     logger\u001b[39m.\u001b[39mdebug(\u001b[39mf\u001b[39m\u001b[39m\"\u001b[39m\u001b[39mSend: \u001b[39m\u001b[39m{\u001b[39;00m_curlify(request)\u001b[39m}\u001b[39;00m\u001b[39m\"\u001b[39m)\n\u001b[1;32m     94\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[0;32m---> 95\u001b[0m     \u001b[39mreturn\u001b[39;00m \u001b[39msuper\u001b[39;49m()\u001b[39m.\u001b[39;49msend(request, \u001b[39m*\u001b[39;49margs, \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n\u001b[1;32m     96\u001b[0m \u001b[39mexcept\u001b[39;00m requests\u001b[39m.\u001b[39mRequestException \u001b[39mas\u001b[39;00m e:\n\u001b[1;32m     97\u001b[0m     request_id \u001b[39m=\u001b[39m request\u001b[39m.\u001b[39mheaders\u001b[39m.\u001b[39mget(X_AMZN_TRACE_ID)\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/requests/adapters.py:644\u001b[0m, in \u001b[0;36mHTTPAdapter.send\u001b[0;34m(self, request, stream, timeout, verify, cert, proxies)\u001b[0m\n\u001b[1;32m    641\u001b[0m     timeout \u001b[39m=\u001b[39m TimeoutSauce(connect\u001b[39m=\u001b[39mtimeout, read\u001b[39m=\u001b[39mtimeout)\n\u001b[1;32m    643\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[0;32m--> 644\u001b[0m     resp \u001b[39m=\u001b[39m conn\u001b[39m.\u001b[39;49murlopen(\n\u001b[1;32m    645\u001b[0m         method\u001b[39m=\u001b[39;49mrequest\u001b[39m.\u001b[39;49mmethod,\n\u001b[1;32m    646\u001b[0m         url\u001b[39m=\u001b[39;49murl,\n\u001b[1;32m    647\u001b[0m         body\u001b[39m=\u001b[39;49mrequest\u001b[39m.\u001b[39;49mbody,\n\u001b[1;32m    648\u001b[0m         headers\u001b[39m=\u001b[39;49mrequest\u001b[39m.\u001b[39;49mheaders,\n\u001b[1;32m    649\u001b[0m         redirect\u001b[39m=\u001b[39;49m\u001b[39mFalse\u001b[39;49;00m,\n\u001b[1;32m    650\u001b[0m         assert_same_host\u001b[39m=\u001b[39;49m\u001b[39mFalse\u001b[39;49;00m,\n\u001b[1;32m    651\u001b[0m         preload_content\u001b[39m=\u001b[39;49m\u001b[39mFalse\u001b[39;49;00m,\n\u001b[1;32m    652\u001b[0m         decode_content\u001b[39m=\u001b[39;49m\u001b[39mFalse\u001b[39;49;00m,\n\u001b[1;32m    653\u001b[0m         retries\u001b[39m=\u001b[39;49m\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mmax_retries,\n\u001b[1;32m    654\u001b[0m         timeout\u001b[39m=\u001b[39;49mtimeout,\n\u001b[1;32m    655\u001b[0m         chunked\u001b[39m=\u001b[39;49mchunked,\n\u001b[1;32m    656\u001b[0m     )\n\u001b[1;32m    658\u001b[0m \u001b[39mexcept\u001b[39;00m (ProtocolError, \u001b[39mOSError\u001b[39;00m) \u001b[39mas\u001b[39;00m err:\n\u001b[1;32m    659\u001b[0m     \u001b[39mraise\u001b[39;00m \u001b[39mConnectionError\u001b[39;00m(err, request\u001b[39m=\u001b[39mrequest)\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/urllib3/connectionpool.py:787\u001b[0m, in \u001b[0;36mHTTPConnectionPool.urlopen\u001b[0;34m(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, preload_content, decode_content, **response_kw)\u001b[0m\n\u001b[1;32m    784\u001b[0m response_conn \u001b[39m=\u001b[39m conn \u001b[39mif\u001b[39;00m \u001b[39mnot\u001b[39;00m release_conn \u001b[39melse\u001b[39;00m \u001b[39mNone\u001b[39;00m\n\u001b[1;32m    786\u001b[0m \u001b[39m# Make the request on the HTTPConnection object\u001b[39;00m\n\u001b[0;32m--> 787\u001b[0m response \u001b[39m=\u001b[39m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49m_make_request(\n\u001b[1;32m    788\u001b[0m     conn,\n\u001b[1;32m    789\u001b[0m     method,\n\u001b[1;32m    790\u001b[0m     url,\n\u001b[1;32m    791\u001b[0m     timeout\u001b[39m=\u001b[39;49mtimeout_obj,\n\u001b[1;32m    792\u001b[0m     body\u001b[39m=\u001b[39;49mbody,\n\u001b[1;32m    793\u001b[0m     headers\u001b[39m=\u001b[39;49mheaders,\n\u001b[1;32m    794\u001b[0m     chunked\u001b[39m=\u001b[39;49mchunked,\n\u001b[1;32m    795\u001b[0m     retries\u001b[39m=\u001b[39;49mretries,\n\u001b[1;32m    796\u001b[0m     response_conn\u001b[39m=\u001b[39;49mresponse_conn,\n\u001b[1;32m    797\u001b[0m     preload_content\u001b[39m=\u001b[39;49mpreload_content,\n\u001b[1;32m    798\u001b[0m     decode_content\u001b[39m=\u001b[39;49mdecode_content,\n\u001b[1;32m    799\u001b[0m     \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mresponse_kw,\n\u001b[1;32m    800\u001b[0m )\n\u001b[1;32m    802\u001b[0m \u001b[39m# Everything went great!\u001b[39;00m\n\u001b[1;32m    803\u001b[0m clean_exit \u001b[39m=\u001b[39m \u001b[39mTrue\u001b[39;00m\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/urllib3/connectionpool.py:464\u001b[0m, in \u001b[0;36mHTTPConnectionPool._make_request\u001b[0;34m(self, conn, method, url, body, headers, retries, timeout, chunked, response_conn, preload_content, decode_content, enforce_content_length)\u001b[0m\n\u001b[1;32m    461\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[1;32m    462\u001b[0m     \u001b[39m# Trigger any extra validation we need to do.\u001b[39;00m\n\u001b[1;32m    463\u001b[0m     \u001b[39mtry\u001b[39;00m:\n\u001b[0;32m--> 464\u001b[0m         \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49m_validate_conn(conn)\n\u001b[1;32m    465\u001b[0m     \u001b[39mexcept\u001b[39;00m (SocketTimeout, BaseSSLError) \u001b[39mas\u001b[39;00m e:\n\u001b[1;32m    466\u001b[0m         \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_raise_timeout(err\u001b[39m=\u001b[39me, url\u001b[39m=\u001b[39murl, timeout_value\u001b[39m=\u001b[39mconn\u001b[39m.\u001b[39mtimeout)\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/urllib3/connectionpool.py:1093\u001b[0m, in \u001b[0;36mHTTPSConnectionPool._validate_conn\u001b[0;34m(self, conn)\u001b[0m\n\u001b[1;32m   1091\u001b[0m \u001b[39m# Force connect early to allow us to validate the connection.\u001b[39;00m\n\u001b[1;32m   1092\u001b[0m \u001b[39mif\u001b[39;00m conn\u001b[39m.\u001b[39mis_closed:\n\u001b[0;32m-> 1093\u001b[0m     conn\u001b[39m.\u001b[39;49mconnect()\n\u001b[1;32m   1095\u001b[0m \u001b[39m# TODO revise this, see https://github.com/urllib3/urllib3/issues/2791\u001b[39;00m\n\u001b[1;32m   1096\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mnot\u001b[39;00m conn\u001b[39m.\u001b[39mis_verified \u001b[39mand\u001b[39;00m \u001b[39mnot\u001b[39;00m conn\u001b[39m.\u001b[39mproxy_is_verified:\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/urllib3/connection.py:753\u001b[0m, in \u001b[0;36mHTTPSConnection.connect\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    751\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[1;32m    752\u001b[0m     sock: socket\u001b[39m.\u001b[39msocket \u001b[39m|\u001b[39m ssl\u001b[39m.\u001b[39mSSLSocket\n\u001b[0;32m--> 753\u001b[0m     \u001b[39mself\u001b[39m\u001b[39m.\u001b[39msock \u001b[39m=\u001b[39m sock \u001b[39m=\u001b[39m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49m_new_conn()\n\u001b[1;32m    754\u001b[0m     server_hostname: \u001b[39mstr\u001b[39m \u001b[39m=\u001b[39m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mhost\n\u001b[1;32m    755\u001b[0m     tls_in_tls \u001b[39m=\u001b[39m \u001b[39mFalse\u001b[39;00m\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/urllib3/connection.py:198\u001b[0m, in \u001b[0;36mHTTPConnection._new_conn\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    193\u001b[0m \u001b[39m\u001b[39m\u001b[39m\"\"\"Establish a socket connection and set nodelay settings on it.\u001b[39;00m\n\u001b[1;32m    194\u001b[0m \n\u001b[1;32m    195\u001b[0m \u001b[39m:return: New socket connection.\u001b[39;00m\n\u001b[1;32m    196\u001b[0m \u001b[39m\"\"\"\u001b[39;00m\n\u001b[1;32m    197\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[0;32m--> 198\u001b[0m     sock \u001b[39m=\u001b[39m connection\u001b[39m.\u001b[39;49mcreate_connection(\n\u001b[1;32m    199\u001b[0m         (\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49m_dns_host, \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mport),\n\u001b[1;32m    200\u001b[0m         \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mtimeout,\n\u001b[1;32m    201\u001b[0m         source_address\u001b[39m=\u001b[39;49m\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49msource_address,\n\u001b[1;32m    202\u001b[0m         socket_options\u001b[39m=\u001b[39;49m\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49msocket_options,\n\u001b[1;32m    203\u001b[0m     )\n\u001b[1;32m    204\u001b[0m \u001b[39mexcept\u001b[39;00m socket\u001b[39m.\u001b[39mgaierror \u001b[39mas\u001b[39;00m e:\n\u001b[1;32m    205\u001b[0m     \u001b[39mraise\u001b[39;00m NameResolutionError(\u001b[39mself\u001b[39m\u001b[39m.\u001b[39mhost, \u001b[39mself\u001b[39m, e) \u001b[39mfrom\u001b[39;00m\u001b[39m \u001b[39m\u001b[39me\u001b[39;00m\n",
      "File \u001b[0;32m~/anaconda3/envs/lcy_clam/lib/python3.10/site-packages/urllib3/util/connection.py:73\u001b[0m, in \u001b[0;36mcreate_connection\u001b[0;34m(address, timeout, source_address, socket_options)\u001b[0m\n\u001b[1;32m     71\u001b[0m \u001b[39mif\u001b[39;00m source_address:\n\u001b[1;32m     72\u001b[0m     sock\u001b[39m.\u001b[39mbind(source_address)\n\u001b[0;32m---> 73\u001b[0m sock\u001b[39m.\u001b[39;49mconnect(sa)\n\u001b[1;32m     74\u001b[0m \u001b[39m# Break explicitly a reference cycle\u001b[39;00m\n\u001b[1;32m     75\u001b[0m err \u001b[39m=\u001b[39m \u001b[39mNone\u001b[39;00m\n",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
     ]
    },
    {
     "ename": "",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m在当前单元格或上一个单元格中执行代码时 Kernel 崩溃。请查看单元格中的代码，以确定故障的可能原因。有关详细信息，请单击 <a href='https://aka.ms/vscodeJupyterKernelCrash'>此处</a>。有关更多详细信息，请查看 Jupyter <a href='command:jupyter.viewOutput'>log</a>。"
     ]
    }
   ],
   "source": [
    "import datasets\n",
    "import pandas as pd\n",
    "\n",
    "# Download labels as csv\n",
    "datasets.load_dataset(\n",
    "    'MahmoodLab/Patho-Bench', \n",
    "    cache_dir='./tutorial-3',\n",
    "    dataset_to_download='cptac_ccrcc',     \n",
    "    task_in_dataset='BAP1_mutation',           \n",
    "    trust_remote_code=True\n",
    ")\n",
    "\n",
    "# Visualize my labels and splits\n",
    "df = pd.read_csv('tutorial-3/cptac_ccrcc/BAP1_mutation/k=all.tsv', sep=\"\\t\")\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check the label distribution\n",
    "df_counts = df['BAP1_mutation'].value_counts().reset_index()\n",
    "df_counts.columns = ['BAP1_mutation', 'Count']\n",
    "df_counts\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### C- Training an ABMIL model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.optim as optim\n",
    "import h5py\n",
    "import pandas as pd\n",
    "from torch.utils.data import Dataset, DataLoader\n",
    "from sklearn.metrics import roc_auc_score\n",
    "\n",
    "from trident.slide_encoder_models import ABMILSlideEncoder\n",
    "\n",
    "# Set deterministic behavior\n",
    "SEED = 1234\n",
    "np.random.seed(SEED)\n",
    "torch.manual_seed(SEED)\n",
    "torch.cuda.manual_seed_all(SEED)\n",
    "torch.backends.cudnn.deterministic = True\n",
    "torch.backends.cudnn.benchmark = False\n",
    "\n",
    "\n",
    "class BinaryClassificationModel(nn.Module):\n",
    "    def __init__(self, input_feature_dim=768, n_heads=1, head_dim=512, dropout=0., gated=True, hidden_dim=256):\n",
    "        super().__init__()\n",
    "        self.feature_encoder = ABMILSlideEncoder(\n",
    "            freeze=False,\n",
    "            input_feature_dim=input_feature_dim, \n",
    "            n_heads=n_heads, \n",
    "            head_dim=head_dim, \n",
    "            dropout=dropout, \n",
    "            gated=gated\n",
    "        )\n",
    "        self.classifier = nn.Sequential(\n",
    "            nn.Linear(input_feature_dim, hidden_dim),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(hidden_dim, 1)\n",
    "        )\n",
    "\n",
    "    def forward(self, x, return_raw_attention=False):\n",
    "        if return_raw_attention:\n",
    "            features, attn = self.feature_encoder(x, return_raw_attention=True)\n",
    "        else:\n",
    "            features = self.feature_encoder(x)\n",
    "        logits = self.classifier(features).squeeze(1)\n",
    "        \n",
    "        if return_raw_attention:\n",
    "            return logits, attn\n",
    "        \n",
    "        return logits\n",
    "\n",
    "# Initialize model\n",
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "model = BinaryClassificationModel().to(device)\n",
    "\n",
    "# Custom dataset\n",
    "class H5Dataset(Dataset):\n",
    "    def __init__(self, feats_path, df, split, num_features=512):\n",
    "        self.df = df[df[\"fold_0\"] == split]\n",
    "        self.feats_path = feats_path\n",
    "        self.num_features = num_features\n",
    "        self.split = split\n",
    "    \n",
    "    def __len__(self):\n",
    "        return len(self.df)\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        row = self.df.iloc[idx]\n",
    "        with h5py.File(os.path.join(self.feats_path, row['slide_id'] + '.h5'), \"r\") as f:\n",
    "            features = torch.from_numpy(f[\"features\"][:])\n",
    "\n",
    "        if self.split == 'train':\n",
    "            num_available = features.shape[0]\n",
    "            if num_available >= self.num_features:\n",
    "                indices = torch.randperm(num_available, generator=torch.Generator().manual_seed(SEED))[:self.num_features]\n",
    "            else:\n",
    "                indices = torch.randint(num_available, (self.num_features,), generator=torch.Generator().manual_seed(SEED))  # Oversampling\n",
    "            features = features[indices]\n",
    "\n",
    "        label = torch.tensor(row[\"BAP1_mutation\"], dtype=torch.float32)\n",
    "        return features, label\n",
    "\n",
    "# Create dataloaders\n",
    "feats_path = './tutorial-3/cptac_ccrcc/20x_512px_0px_overlap/features_conch_v15'\n",
    "batch_size = 8\n",
    "train_loader = DataLoader(H5Dataset(feats_path, df, \"train\"), batch_size=batch_size, shuffle=True, worker_init_fn=lambda _: np.random.seed(SEED))\n",
    "test_loader = DataLoader(H5Dataset(feats_path, df, \"test\"), batch_size=1, shuffle=False, worker_init_fn=lambda _: np.random.seed(SEED))\n",
    "\n",
    "# Training setup\n",
    "criterion = nn.BCEWithLogitsLoss()\n",
    "optimizer = optim.Adam(model.parameters(), lr=4e-4)\n",
    "\n",
    "# Training loop\n",
    "num_epochs = 20\n",
    "for epoch in range(num_epochs):\n",
    "    model.train()\n",
    "    total_loss = 0.\n",
    "    for features, labels in train_loader:\n",
    "        features, labels = {'features': features.to(device)}, labels.to(device)\n",
    "        optimizer.zero_grad()\n",
    "        outputs = model(features)\n",
    "        loss = criterion(outputs, labels)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "        total_loss += loss.item()\n",
    "    print(f\"Epoch {epoch+1}/{num_epochs}, Loss: {total_loss/len(train_loader):.4f}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### D- Evaluating the ABMIL model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Evaluation\n",
    "model.eval()\n",
    "all_labels, all_outputs = [], []\n",
    "correct = 0\n",
    "total = 0\n",
    "\n",
    "with torch.no_grad():\n",
    "    for features, labels in test_loader:\n",
    "        features, labels = {'features': features.to(device)}, labels.to(device)\n",
    "        outputs = model(features)\n",
    "        \n",
    "        # Convert logits to probabilities and binary predictions\n",
    "        predicted = (outputs > 0).float()  # Since BCEWithLogitsLoss expects raw logits\n",
    "        correct += (predicted == labels).sum().item()\n",
    "        total += labels.size(0)\n",
    "\n",
    "        all_outputs.append(outputs.cpu().numpy())  \n",
    "        all_labels.append(labels.cpu().numpy())\n",
    "\n",
    "# Compute AUC\n",
    "all_outputs = np.concatenate(all_outputs)\n",
    "all_labels = np.concatenate(all_labels)\n",
    "auc = roc_auc_score(all_labels, all_outputs)\n",
    "\n",
    "# Compute accuracy\n",
    "accuracy = correct / total\n",
    "print(f\"Test AUC: {auc:.4f}\")\n",
    "print(f\"Test Accuracy: {accuracy:.4f}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### E- Extract attention heatmap for the freshly trained model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from trident import OpenSlideWSI, visualize_heatmap\n",
    "from trident.segmentation_models import segmentation_model_factory\n",
    "from trident.patch_encoder_models import encoder_factory as patch_encoder_factory\n",
    "\n",
    "# a. Load WSI to process\n",
    "job_dir = './tutorial-3/heatmap_viz'\n",
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "slide = OpenSlideWSI(slide_path='./CPTAC-CCRCC_v1/CCRCC/C3L-00418-22.svs', lazy_init=False)\n",
    "\n",
    "# b. Run segmentation \n",
    "segmentation_model = segmentation_model_factory(\"hest\")\n",
    "geojson_contours = slide.segment_tissue(segmentation_model=segmentation_model, job_dir=job_dir, device=device)\n",
    "\n",
    "# c. Run patch coordinate extraction\n",
    "coords_path = slide.extract_tissue_coords(\n",
    "    target_mag=20,\n",
    "    patch_size=512,\n",
    "    save_coords=job_dir,\n",
    "    overlap=256, \n",
    ")\n",
    "\n",
    "# d. Run patch feature extraction\n",
    "patch_encoder = patch_encoder_factory(\"conch_v15\").eval().to(device)\n",
    "patch_features_path = slide.extract_patch_features(\n",
    "    patch_encoder=patch_encoder,\n",
    "    coords_path=coords_path,\n",
    "    save_features=os.path.join(job_dir, f\"features_conch_v15\"),\n",
    "    device=device\n",
    ")\n",
    "\n",
    "#  e. Run inference \n",
    "with h5py.File(patch_features_path, 'r') as f:\n",
    "    coords = f['coords'][:]\n",
    "    patch_features = f['features'][:]\n",
    "    coords_attrs = dict(f['coords'].attrs)\n",
    "\n",
    "batch = {'features': torch.from_numpy(patch_features).float().to(device).unsqueeze(0)}\n",
    "logits, attention = model(batch, return_raw_attention=True)\n",
    "\n",
    "# f. generate heatmap\n",
    "heatmap_save_path = visualize_heatmap(\n",
    "    wsi=slide,\n",
    "    scores=attention.cpu().numpy().squeeze(),  \n",
    "    coords=coords,\n",
    "    vis_level=1,\n",
    "    patch_size_level0=coords_attrs['patch_size_level0'],\n",
    "    normalize=True,\n",
    "    num_top_patches_to_save=10,\n",
    "    output_dir=job_dir\n",
    ")\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "trident",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
